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Introduction 


Campuses that pursue the Carnegie Community Engagement 
Classification will in some form take full inventory of their engagement 
efforts in order to address the range of questions posed by the Carnegie 
Foundation. (Thornton & Zuiches, 2009, p. 75). 


The Community Engagement Classification is an elective classification offered by the 
Carnegie Foundation for the Advancement of Teaching. In order to be classified, 
campuses provide evidence documenting engagement through an application process. 
Campuses were classified in 2006, 2008, and 2010, and will be classified on five-year 
cycles from 2015 onward. (Information about the classification can be found on the 


Carnegie Foundation website.) 


This mixed-methods, two-part study sought to discover how institutions that received the 
Carnegie Community Engagement Classification in 2010 approached their application 
process and to examine the longer term outcomes of that process. How did they 
undertake a “full inventory” of their community engagement efforts, and what were the 
outcomes of undertaking such an inventory, beyond receiving the Classification? 


The study had two purposes: (a) To document the strategies and methods used by 
successful applicants for the 2010 Carnegie Community Engagement Classification, and 
(b) to document the cultural shifts experienced by institutions as they developed their 
successful applications for this Classification. 


Methods and Data Sources 


The study took place during 2012-2013 in two phases: 


Phase I. Using a list of the 121 institutions that received the Classification in 2010, the 
first author selected nine applicants representing a range of student-body size, Carnegie 
institution type, and geographic area. She conducted 30-minute, semi-structured, 11- 
question interviews with these nine applicants’ and used those responses to refine a 
survey for the full set of 2010 classified institutions. 


Phase Il. The authors then prepared a more robust 21-question survey with both 
structured and semi-structured elements in order to more effectively map institutional 
characteristics against the processes, participants, and motivations discovered through 
the interviews. The authors conducted the survey using SurveyMonkey and contacted 
the entire set of 121 successful applicants. Fifty-two applicants agreed to participate, a 
43% response rate. The 52 institutions completing the survey represented a cross- 
section of institutional type and size. 


"We have used the term “applicants” to describe the interviewees and respondents. While the Carnegie Community 
Engagement Classification application asks for contact names and information, sometimes those contacts are not the 
authors of the applications but rather administrators to whom general questions might be directed. For this study, 
each interviewee or respondent confirmed that he or she was indeed the author, described as “applicant” in this 
paper. 
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Institution's Carnegie Classification 


= Masters 

= Research (high/very high level of 
research) 

= Doctoral/Research 


= Baccalaureate 


= Associates 


Figure 1. Institutional Demographics: Carnegie Classification 


Public or Private 


= Public 


= Private 


Figure 2. Institutional Demographics: Public or Private 


Student Enrollment 


= Fewer than 5,000 students 


= 10,001-20,000 students 


= 5,001-10,000 students 


= More than 20,000 students 


Figure 3. Institutional Demographics: Student Enrollment 
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Limitations of the Study 


The study design allowed a self-selected sampling. While all 121 institutions that 
received the Classification in 2010 were invited to participate, all institutions had the 
option of declining to complete the survey. The result of this design was an oversampling 
of certain types of institutions—that is, the percentage of applicants from each category 
(masters, research, associates, etc.) does not equal the actual percentage of institutions 
from that category that received the 2010 Classification. 


This study focused entirely on the application process and perceptions of institutional 
change that occurred as a result of applying for the 2010 Classification. Since the study 
was conducted through interviews and surveys, all data are self-reported. No attempts 
were made to verify or confirm the applicants’ responses, and the results do not 
demonstrate the quality of community engagement at these institutions. Further, while 
the study gives a snapshot of the 2010 classified institutions, it does not chart the 
institutions’ longitudinal community engagement. 


Study Results 


Leadership in Community Engagement: Application Lead Authors 


This study found a wide variability in the titles and positions of the Carnegie 
Classification application lead authors. Of the 52 institutions included in the study, 
applicants held 28 different positions/titles. There were 21 Directors/Coordinators at the 
institutional level (i.e., Director of Outreach and Engagement), 16 Directors/Coordinators 
of centers, with the remainder of authors being faculty, project directors, etc. 
Additionally, the titles of the applicants included multiple terms to designate how their 
respective institution describes its work with the community. Terms repeated across 
positions/titles included: Civic Engagement, Community Partnerships, Institutional 
Effectiveness, Research, Community Engagement, Community-Based Learning, 
Outreach and Engagement, and Service Learning. 


Institutional Unit of Application Lead Authors 


Academic Affairs 

A Center or Institute 
Student Affairs 

A Department 
Institutional Research 
President's Office 


Figure 4. Institutional Unit of Application Lead Authors 
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Application Teams 


Another key strategy undertaken by applicants was to work with a team. As Driscoll 
(2009) explains, “The documentation process is intensive and requires the collaboration 
of many institutional and community participants” (p. 7). 


This study found application teams comprised an average of 6.1 members. Responses 
were generally evenly distributed around three methods that the institutions used to 
select the members of the application and data-gathering team: (a) A recognized 
campus leader in community engagement initiated a team, (b) a president or other 
cabinet member identified team members, and (c) a community engagement team 
already existed and continued their work. 


Selection of Application Team 


mA recognized 
campus leader in 
community 
engagement 
initiated team 


= President or other 
Cabinet member 
identified team 
members 


= A community 
engagement team 
was already in 
place and 
continued their 
work 


Figure 5. Selection of Application Team 


Application Roles of Institutional and Community Members 


While the application lead author and application team play key roles in preparing the 
application, many groups representing a variety of institutional and community 
perspectives must be consulted when working to document such a cross-institutional 
practice as community engagement. 


e The main role for faculty, students, community partners, office or center directors, 
deans, and department chairs was to contribute information in the form of 
completing surveys, serving as interviewees, or serving as members of focus 
groups. 

e Staff were the most frequently identified members of data-gathering and 
application preparation teams. 

e Upper-level administrators most commonly provided feedback on drafts of the 
application. 
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Data Sources 


The application for the Classification requires a wide range of data regarding community 
engagement across multiple units and constituencies. In some cases, the data needed 
to complete the Carnegie application already exist, having been gathered previously for 
other purposes. In some cases, however, new data need to be gathered to complete all 
of the questions on the application. Applicants indicated that, on average, 62% of the 
data needed to complete the application were found to already have existed on their 
campuses. The remaining 38% of the data were newly gathered specifically for the 
application process. 


Table 1. Data Sources: Existing and Newly Collected 


Data Type Percentage of Data Number of Data 
Used in Application Sources 
Existing Data 62% 22 
Newly Gathered Data 38% 17 
Existing Data Sources Utilized Newly Collected Data Sources 
Reported by Multiple Applicants Reported by Multiple Applicants 


e Self-studies Interviews and one-on-one conversations 
e Colleges (including their web sites) Surveys 
e Center or Office for Community or Civic E-mail requests 


e 
e 
e 
Engagement e University web sites 
e 
e 
e 


Faculty publications Community members 

e NSSE/FSSE reports Created new database (i.e. Banner) 
Existing data re-collected 

e Focus groups 


Reported by Single Applicants Reported by Single Applicants 
e Advisory groups and councils e Annual reports 
e Annual reports e Data base (new data gathered) 
e Campus Compact surveys e Departments and units — personal contact 
e Cooperative Extension e Financial records 
e Departments e Focus groups 
e Extended university data e Handbooks and policy manuals 
e Fact Books e Institutional Research 
e Grant-funded programs e Press releases 
e Historical documents e University publications 
e Institutional Research 
e Library database 
e Main student service and volunteerism unit 
e Previous applications for awards and grants 
e Previous application for President’s Honor 


Rolls 

Registrar data 

Student services administrative offices 
“Researched and gathered information on 
existing commitments, policies, procedures, 
and documented philosophy.” 


NERCHE Working Paper: “Gathering Data and Documenting Impact” Page 6 


Application Preparation Time 


Overall, the average amount of time spent on the application was 6.6 months. 
Applicants reported spending an average of 29% of their work time during those months 
preparing the application. 


Table 2. Length of Time to Prepare Application 


Months Spent on Application: % of Time Spent on 
Average Application: Average 
6.6 29% 
Months Spent on Application # of Students 
5.8 Fewer than 5,000 
7 5,001-10,000 
7.5 10,000-20,000 
9.1 More than 20,000 


As seen in Table 2, there appears to be a direct correlation between institutional size, 
measured in number of students, and months spent on the application. Two interviewees 
from Phase | of the study posited that the smaller the institution, the less time is needed 
to complete the application. One interviewee stated that it is easier at a small institution 
because “We know who does what, who is motivated.” The other indicated that at the 
smaller institution, “| know everyone who does this work,” and the institution is “compact 
enough to keep track of.” Alternative explanations for the length of time needed at 
different-sized institutions may be availability of resources for the task or robustness of 
an institution’s designated Community Engagement Center. Further research is needed 
to advance our understanding of this particular issue. 


Additional Support Received for Work 


The majority of applicants tapped into existing units, documents, or workshops in 
completing their applications. In particular, respondents indicated the benefits of 
reviewing the applications of previously successful Carnegie classified institutions as 
well as attending webinars or workshops provided by the Carnegie Foundation. A very 
small percentage received support in the form of additional funding, additional staff, 
students, payment, or release time, while a few applicants expressed a concern over a 
lack of support for the work. 
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Additional Support Received for Work on Application 
76.6% 


68.1% 


Financial Offices or Additional staff Viewed Attended a None 
Support (e.g. centers (e.g. or students to another Carnegie 
payment or §= Community assist in institution's Community 
release time) Engagement, process successful Engagement 
Institutional application Classification 
Research) workshop 


Figure 6. Additional Support Received for Work on Application 


Upper Level Administrator Involvement 


A large majority of applicants indicated that their upper-level administration was involved 
in communicating about the Classification, either about the importance of participating in 
the process or about receiving the Classification. However, less than 15% responded 
that upper-level administrators authored the application, provided monetary or release- 
time support for the application lead author, or created a new or filled a vacant position 
for the purpose of the application. 


Upper Level Administrative Support 


Made an announcement to the campus and/or 
community about receiving the Classification 


Alerted campus to the importance of 
participating in the Classification process 


Created a new (or filled a vacant) faculty or staff 
position in support of the application process 


Provided monetary or release time support for 
author(s) of theapplication 


Helped develop the strategy for gathering data 
for theapplication 


Authored the application 


Figure 7. Upper Level Administrative Support 
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Challenges or Obstacles Faced in Completing the Application 


Applicants encountered a number of challenges during the application preparation, with 
the five main issues including: 


No structure for data collection in place prior to beginning the application process. 
o As five respondents elaborated, it was not so much that they had no structure, 
but rather minimal structure. As one interviewee described, data collection 
was not “comprehensive” prior to the application process, and another 

described it as “spotty.” 


Difficulty ensuring institution-wide involvement. 

o Examples of this difficulty included reports that key administrators “grumbled” 
or showed “indifference” to the Classification itself. However, as one 
respondent described, “Once we figured out who to talk to, people were very 
helpful.” 


Definition of “community engagement” varied or was unclear at the institution. 

o Responses ranged from an interviewee who indicated that colleagues did not 
see how community engagement was part of the institution’s mission, to a 
survey respondent who explained that many units wanted to have their work 
counted as community engagement, even if it did not fit the accepted 
Carnegie definition. 


Insufficient resources/time. 

o One-third of the respondents indicated they needed more time, resources, 
and support to complete their applications. 

Difficulty matching responses to the wording on the application. 

o Difficulties included problems fitting responses into the word or character limit 
required by the application and in aligning the data with the questions on the 
application. 


Challenges or Obstacles 


62.2% 
55.6% 


No structure for _ Difficulty ensuring Definition of Insufficient Difficulty matching 
data collection in institution-wide "community resources/time responses to 


place prior to involvement engagement" wording on 
beginning unclear at my application 


application process institution 


Figure 8. Challenges or Obstacles 
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Post-Classification, or Now What? 
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Changes in Institutional Culture as a Result of Applying for or Receiving the 
Classification 


In the study survey, many applicants indicated progress on institutionalizing community 
engagement during the application process. This progress ranged from improved 
collaborations and greater involvement of on-campus groups, to improved data 
structures, to increased support for community engagement. Applicants also reported 
attempts to align community engagement with the mission of the university, and the 
converse, “trying to transform into the Engaged University.” These findings are 
consistent with Sandmann, Thornton, and Jaeger (2009), who note that “The Carnegie 
community engagement process and its data can also serve as a vehicle for 
institutionalizing engagement” (p. 4). Importantly, some changes were noted as a result 
of applying for, as well as of receiving, the Classification. 


Table 4. Changes in Institutional Culture upon Receiving Classification 


Changes in Institutional Culture % of Respondents 
New, increased, or improved cross-campus collaborations 71% 
Greater involvement by administration/faculty/staff/students/ community in 69% 
institutionalizing community engagement ° 
Better alignment of institution’s mission with goals of community 48%. 
engagement i 
New or improved data reporting structures for community engagement 56% 
New, increased, or improved partnership with community 52% 


Structural changes in university to support community engagement (i.e. 
new positions or assignments of faculty/staff/administration in order to 36% 
support campus-wide community engagement) 


Plan for Announcing the Classification 


Forty-four percent of applicants indicated that they had prepared a plan for how to 
announce and celebrate the receipt of the Classification, describing their work in 
partnership with the marketing unit within their institution, such as communications, 
public relations, or marketing. A number of platforms were utilized for announcing receipt 
of the Classification. 


Announcements in local media 
e Announcements to external colleagues and institutions (“We printed 
announcements and sent them out to colleagues, peer institutions, and sister 
institutions.”) 
Announcements to various stakeholder groups 
Award-related events 
Banner on website 
Banners placed on campus and at campus entrances 
Campus celebration (“We held a reception in which community partners, faculty, 
students, and the college community was invited to celebrate with us. There was 
media attention as well.”) 
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Facebook 

Internal publications (e.g., university magazine, etc.) 

Mailers to US News and World Report ranking input providers 
Newspaper articles 

President's newsletter 

Press releases 

Web sites 


Applicant Recommendations to Future Classification Applicants 


Most interviewees offered three to five recommendations for future Community 
Engagement Classification applicants. These recommendations can be grouped into 12 
categories, presented below in order of number of recommendations offered: 


Form a Team (25) 

Utilize or Develop a Data-Gathering Structure (12) 

Gain Administrator Involvement and/or Support (11) 

Institutionalize or Centralize Service-Learning, Community Engagement, or 
Research (9) 

Generate Awareness of the Classification (8) 

Use Multiple Sources of Data and Resources (8) 

Start Early (7) 

Adopt a Campus-wide Definition of Community Engagement and Service 
Learning (5) 

Identify a Single Lead Author (4) 

Tie Data Gathering Directly to Application (4) 

Use Previously Gathered Data or Tie to Other Initiatives (4) 

Attend Workshops or Work with Successfully Classified Institutions (3) 
Tie Mission to Service (3) 


Conclusion and Recommended Future Research 


Community engagement has become a valued practice in higher education, though it 
remains diffuse, with evidence of the practice and its impacts spread throughout an 
institution. The Carnegie Community Engagement Classification provides the opportunity 
for institutions to gather data and present evidence of their institutional commitment to 
community engagement. In this study of 52 U.S. institutions that successfully received 
the 2010 Classification, applicants held in common a number of key observations and 
strategies related to the application process: 


1. 


Successful applicants utilized a team approach to gathering the data and 
completing the application. 

Evidence of community engagement often already exists within an institution 
(62% of the Classification application utilized existing data), but some new data 
will likely need to be gathered (38% of data utilized was newly gathered). 
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3. There were several obstacles or challenges experienced by successful 
applicants related to data collection structures and resources, institution-wide 
understanding and involvement, and matching data to the application itself. 


4. Institutional and cultural shifts were identified by over half of the applicants, not 
only upon receipt of the classification, but also simply by engaging in the 
application process. Cultural shifts included new or improved collaborations, 
greater institutionalization of community engagement, new or improved data 
reporting structures, and better alignment of the institution’s mission with the 
goals of community engagement. 


The Carnegie Community Engagement Classification, and community engagement in 
general, remains a fruitful field for study. Future research might include the following: 


1. A longitudinal study of any long-lasting effects of applying for or receiving the 
Carnegie Community Engagement Classification. 

2. A study of the quality of community engagement at Carnegie classified 
institutions, using such measurement tools as self-reports on scales or rubrics. 

3. A comparison of successful and unsuccessful Classification applications, studied 
for the purpose of determining if there are any clear strategies that enable 
successful applications or notable gaps that frequently lead to an unsuccessful 
submission. 
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